Structure-Aware Dynamic Scheduler for Parallel Machine Learning

Training large machine learning (ML) models with many variables or parameters can take a long time if one employs sequential procedures even with stochastic updates. A natural solution is to turn to distributed computing on a cluster; however, naive, unstructured parallelization of ML algorithms does not usually lead to a proportional speedup and can even result in divergence, because dependencies between model elements can attenuate the computational gains from parallelization and compromise correctness of inference. Recent efforts toward this issue have benefited from exploiting the static, a priori block structures residing in ML algorithms. In this paper, we take this path further by exploring the dynamic block structures and workloads therein present during ML program execution, which offers new opportunities for improving convergence, correctness, and load balancing in distributed ML. We propose and showcase a general-purpose scheduler, STRADS, for coordinating distributed updates in ML algorithms, which harnesses the aforementioned opportunities in a systematic way. We provide theoretical guarantees for our scheduler, and demonstrate its efficacy versus static block structures on Lasso and Matrix Factorization

Luminosity Evolution of Early-type Galaxies to z=0.83: Constraints on Formation Epoch and Omega

We present deep spectroscopy with the Keck telescope of eight galaxies in the luminous X-ray cluster MS1054-03 at z=0.83. The data are combined with imaging observations from the Hubble Space Telescope (HST). The spectroscopic data are used to measure the internal kinematics of the galaxies, and the HST data to measure their structural parameters. Six galaxies have early-type spectra, and two have "E+A" spectra. The galaxies with early-type spectra define a tight Fundamental Plane (FP) relation. The evolution of the mass-to-light ratio is derived from the FP. The M/L ratio evolves as \Delta log M/L_B \propto -0.40 z (Omega_m=0.3, Omega_Lambda=0). The observed evolution of the M/L ratio provides a combined constraint on the formation redshift of the stars, the IMF, and cosmological parameters. For a Salpeter IMF (x=2.35) we find that z_form>2.8 and Omega_m<0.86 with 95% confidence. The constraint on the formation redshift is weaker if Omega_Lambda>0: z_form>1.7 if Omega_m=0.3 and Omega_Lambda=0.7. At present the limiting factor in constraining z_form and Omega from the observed luminosity evolution of early-type galaxies is the poor understanding of the IMF. We find that if Omega_m=1 the IMF must be significantly steeper than the Salpeter IMF (x>2.6).Comment: To be published in ApJ Letters, Volume 504, September 1, 1998. 5 pages, 4 figure

A Database of Cepheid Distance Moduli and TRGB, GCLF, PNLF and SBF Data Useful for Distance Determinations

We present a compilation of Cepheid distance moduli and data for four secondary distance indicators that employ stars in the old stellar populations: the planetary nebula luminosity function (PNLF), the globular cluster luminosity function (GCLF), the tip of the red giant branch (TRGB), and the surface brightness fluctuation (SBF) method. The database includes all data published as of July 15, 1999. The main strength of this compilation resides in all data being on a consistent and homogeneous system: all Cepheid distances are derived using the same calibration of the period-luminosity relation, the treatment of errors is consistent for all indicators, measurements which are not considered reliable are excluded. As such, the database is ideal for inter-comparing any of the distance indicators considered, or for deriving a Cepheid calibration to any secondary distance indicator. Specifically, the database includes: 1) Cepheid distances, extinctions and metallicities; 2) apparent magnitudes of the PNLF cutoff; 3) apparent magnitudes and colors of the turnover of the GCLF (both in the V- and B-bands); 4) apparent magnitudes of the TRGB (in the I-band) and V-I colors at and 0.5 magnitudes fainter than the TRGB; 5) apparent surface brightness fluctuation magnitudes I, K', K_short, and using the F814W filter with the HST/WFPC2. In addition, for every galaxy in the database we give reddening estimates from DIRBE/IRAS as well as HI maps, J2000 coordinates, Hubble and T-type morphological classification, apparent total magnitude in B, and systemic velocity. (Abridged)Comment: Accepted for publication in the Astrophysical Journal Supplement Series. Because of space limitations, the figures included are low resolution bitmap images. Original figures can be found at http://www.astro.ucla.edu/~laura/pub.ht

The HST Key Project on the Extragalactic Distance Scale XXV. A Recalibration of Cepheid Distances to Type Ia Supernovae and the Value of the Hubble Constant

Cepheid-based distances to seven Type Ia supernovae (SNe)-host galaxies have been derived using the standard HST Key Project on the Extragalactic Distance Scale pipeline. For the first time, this allows for a transparent comparison of data accumulated as part of three different HST projects, the Key Project, the Sandage et al. Type Ia SNe program, and the Tanvir et al. Leo I Group study. Re-analyzing the Tanvir et al. galaxy and six Sandage et al. galaxies we find a mean (weighted) offset in true distance moduli of 0.12+/-0.07 mag -- i.e., 6% in linear distance -- in the sense of reducing the distance scale, or increasing H0. Adopting the reddening-corrected Hubble relations of Suntzeff et al. (1999), tied to a zero point based upon SNe~1990N, 1981B, 1998bu, 1989B, 1972E and 1960F and the photometric calibration of Hill et al. (1998), leads to a Hubble constant of H0=68+/-2(random)+/-5(systematic) km/s/Mpc. Adopting the Kennicutt et al. (1998) Cepheid period-luminosity-metallicity dependency decreases the inferred H0 by 4%. The H0 result from Type Ia SNe is now in good agreement, to within their respective uncertainties, with that from the Tully-Fisher and surface brightness fluctuation relations.Comment: Accepted for publication in The Astrophysical Journal. 62 pages, LaTeX, 9 Postscript figures. Also available at http://casa.colorado.edu/~bgibson/publications.htm

The HST Key Project on the Extragalactic Distance Scale. XXVIII. Combining the Constraints on the Hubble Constant

Since the launch of the Hubble Space Telescope nine years ago, Cepheid distances to 25 galaxies have been determined for the purpose of calibrating secondary distance indicators. A variety of these can now be calibrated, and the accompanying papers by Sakai, Kelson, Ferrarese, and Gibson employ the full set of 25 galaxies to consider the Tully-Fisher relation, the fundamental plane of elliptical galaxies, Type Ia supernovae, and surface brightness fluctuations. When calibrated with Cepheid distances, each of these methods yields a measurement of the Hubble constant and a corresponding measurement uncertainty. We combine these measurements in this paper, together with a model of the velocity field, to yield the best available estimate of the value of H_0 within the range of these secondary distance indicators and its uncertainty. The result is H_0 = 71 +/- 6 km/sec/Mpc. The largest contributor to the uncertainty of this 67% confidence level result is the distance of the Large Magellanic Cloud, which has been assumed to be 50 +/- 3 kpc

The HST Key Project on the Extragalactic Distance Scale XXVI. The Calibration of Population II Secondary Distance Indicators and the Value of the Hubble Constant

A Cepheid-based calibration is derived for four distance indicators that utilize stars in the old stellar populations: the tip of the red giant branch (TRGB), the planetary nebula luminosity function (PNLF), the globular cluster luminosity function (GCLF) and the surface brightness fluctuation method (SBF). The calibration is largely based on the Cepheid distances to 18 spiral galaxies within cz =1500 km/s obtained as part of the HST Key Project on the Extragalactic Distance Scale, but relies also on Cepheid distances from separate HST and ground-based efforts. The newly derived calibration of the SBF method is applied to obtain distances to four Abell clusters in the velocity range between 3800 and 5000 km/s, observed by Lauer et al. (1998) using the HST/WFPC2. Combined with cluster velocities corrected for a cosmological flow model, these distances imply a value of the Hubble constant of H0 = 69 +/- 4 (random) +/- 6 (systematic) km/s/Mpc. This result assumes that the Cepheid PL relation is independent of the metallicity of the variable stars; adopting a metallicity correction as in Kennicutt et al. (1998), would produce a (5 +/- 3)% decrease in H0. Finally, the newly derived calibration allows us to investigate systematics in the Cepheid, PNLF, SBF, GCLF and TRGB distance scales.Comment: Accepted for publication in the Astrophysical Journal. 48 pages (including 13 figures and 4 tables), plus two additional tables in landscape format. Also available at http://astro.caltech.edu/~lff/pub.htm K' SBF magnitudes have been update